1. Field of the Invention
The present invention relates to an image processing apparatus and an image processing method that corrects brightness of an input image.
2. Description of the Related Art
Conventionally, various methods to execute various corrections to photo image data taken by a digital camera to display and print a suitable photo image have been discussed. Such an image correction method can be roughly divided into two methods. One is a uniform correction that executes a uniform correction on an entire image and the other is a local correction in which a correction amount changes according to local properties of an image.
Examples of the former uniform correction include a color balance correction as discussed in Japanese Patent Application Laid-Open No. 2000-013616. In Japanese Patent Application Laid-Open No. 2000-013616, a luminance histogram of an image is calculated and a highlight/shadow point in the image is calculated according to cumulative frequency. If a color difference component of the highlight/shadow point is 0, the color balance of the image data is determined to be appropriate. If the color difference component of the highlight/shadow point is not 0, on the other hand, the color balance is determined to be lost and a rotation matrix that makes the color difference of the highlight/shadow point 0 in the luminance/color difference space is uniformly applied to the image.
Regarding the latter local correction, on the other hand, examples include image processing of a type, as typified by a dodging process, in which the correction amount changes according to a spatial position of the image. The dodging process is, for example, if a subject such as a person is dark and the background is bright, to correct brightness of a person area appropriately while suppressing overexposure of the background by significantly increasing luminance of the dark person area and changing the luminance of the bright background area not so much. Japanese Patent Application Laid-Open No. 2001-298619 and Japanese Patent No. 3233114 have discussed such a dodging process. Both realize the dodging process in digital processing by generating a low-frequency image, that is, a blurred image by performing filtering on an input image and using the blurred image as a spatial brightness control factor.
Printers in recent years have various functions and are provided with, in addition to printing, a copy function and a function to directly print photo data from a memory card as standard functions. Further, enhancement in a printing speed in recent years has also been remarkable and continuous enhancement has been demanded for the future. On the other hand, hardware resources mounted in a printer have not been as abundant as those on a personal computer in recent years. A central processing unit (CPU) speed, a random access memory (RAM) capacity, and an access speed of the RAM have generally limited capabilities. Further, demands for cost reduction of a printer main body are severe, and thus image processing that can operate with as low resources as possible has been desired. In other words, hardware performance of a printer main body has been confronted with a mutually contradictory situation in which performance improvement of the printer speed and the like has been desired while capabilities thereof being maintained or reduced.
Under such circumstances, image processing using a three-dimensional lookup table (hereinafter, referred to as a 3DLUT) is discussed as an effective unit for performing image processing at a high speed. Effectiveness of image processing using the 3DLUT system will be described below. For example, a case where a color balance correction, luminance gradation correction, and saturation correction like those discussed in Japanese Patent Application Laid-Open No. 2000-013616 are performed simultaneously on an input image is considered. A calculation amount when the above processing is performed on each of all pixels of the entire image and that when pixels of the entire image are corrected by interpolation calculation by constituting a corrected red-green-blue (RGB) 3DLUT.
First, the former case where processing is performed on each of the entire image is considered. If each pixel value of the input image is represented in RGB, the pixel value is converted into, for example, YCbCr values which are a luminance color difference space. This processing requires nine multiplications. A three-dimensional rotation matrix for correction as discussed in Japanese Patent Application Laid-Open No. 2000-013616 is applied to the YCbCr values to obtain Y′Cb′Cr′ values after the color balance correction. This processing requires nine multiplications. Next, a predetermined gradation correction is performed on the Y′ value to obtain a Y″ value. Further, the Cb′Cr′ values are multiplied by a saturation enhancement coefficient to obtain Cb′Cr′ values. This processing requires two multiplications. Lastly, the corrected Y″Cb″Cr″ values are converted into R″G″B″ values using a publicly known formula. This processing requires nine multiplications. Therefore, this sequence of processing requires a total of 29 multiplications. If this processing is repeated for each pixel, 174 million multiplications are needed for an image of, for example, 3000*2000 pixels.
A case where, on the other hand, a 3DLUT is used will be described. As is well known, according to a system using the 3DLUT, the 3DLUT is divided into a grid pattern having predetermined intervals therein and the corrected RGB values are calculated for each grid point and stored. It is assumed that, for example, the grid interval is 32 and 9^3=729 grid points are present. If the above processing is performed for the RGB values at each grid point, 29*729 multiplications are needed. The 3DLUT generated in this manner is applied to each pixel by publicly known interpolation calculations. If, for example, tetrahedral interpolation is used, corrected RGB values can be obtained by five multiplications per pixel. More specifically, the number of multiplications by the 3DLUT for the entire image is represented by the following formula:Number of multiplications=29*729+(3000*4000)*5=about 60.02 million
Comparison of total multiplications of both shows that the number of calculations of the latter is about ⅓ of that of the former so that the 3DLUT system turns out to be extremely effective.
On the other hand, among photo images captured by an ordinary amateur photographer who is not a professional photographer, there are many images including a plurality of failure factors such as a backlight photo with color fog. For such images, the color fog is removed by the color balance correction and then, very complex correction processing, such as increasing a luminance component for only a dark subject region while leaving the background as it is, is needed.
Considering that a local correction as typified by a dodging process correction and a uniform correction throughout an image like the color balance correction are performed simultaneously, there is an issue that it is very difficult to control using only one 3DLUT like the above one.
The 3DLUT is applied, as described above, by interpolation calculations and thus, for example, extracting only a brightness component from interpolated pixel values to locally control the component may be conceivable, but processing is made thereby more complex which increases scale of calculations and that of circuits necessary for realization.